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TITLE: MODULAR SENSOR ARRAY 

INVENTOR: DAVID A. MONROE 



BACKGROUND OF INVENTION: 
[0001 ] 1 . FIELD OF INVENTION: This invention is generally related to field surveillance 

equipment and is specifically directed to a multi-function, modular, portable field system having 
common control, power and management electronics for a plurality of distinct surveillance 
module units. 

[0002] 2. DESCRIPTION OF THE PRIOR ART: Military scouts and other personnel who 

are tasked to perform surveillance and reconnaissance operations must deal with widely varying 
conditions. These include different types of subject material to observe, differing environmental 
m conditions such as lighting, humidity, temperature, dust and pollution, all which may adversely 
^ impact surveillance, and threat conditions such as terrain, water, chemical hazards, radiation 
™ hazards, or hostile military or civilian attacks. 

[0003] A variety of sensors are currently being utilized by scouts in order to accommodate the 

conditions and situations that they encounter. Daylight Cameras, Image Intensifies, uncooled 
FLIR (Forward Looking Infrared) systems, Laser Rangefinders, RF Sensors, GPS receivers, NBC 

Q Detectors (Nuclear, Biological, Chemical Detectors), and other types of sensors are currently 
being utilized for surveillance and reconnaissance operations. It is also desired to utilize 
cryogenic FLIR systems, which are currently too cumbersome and noisy to man-carry. 

[0004] Surveillance personnel may have to walk great distances to get in an ideal position to 

perform surveillance. The weight of the system required becomes an important factor to the 
scout. Also the number of different types of devices that may be required to perform the 
spectrum of surveillance required can increase the scout's load and the complexity of the task, 
requiring the operator to understand the details of operation of a diverse assemblage of disjointed 
units. Further, powering a fleet of diverse units all independently designed requires a variety of 
different types of batteries, thus generating more cost, weight and confusion. 

[0005] Military scouts and other personnel who are tasked to perform surveillance and 

reconnaissance operations must deal with widely varying conditions. These include different 
types of subject material to observe, differing environmental conditions such as lighting, 
humidity, temperature, dust and pollution all which may adversely impact surveillance, and threat 



2/35 

conditions such as terrain, water, chemical hazards, radiations hazards, or hostile military or 
civilian attacks. 

[0006] Finally, the output of the sensors has traditionally been viewed by a human. It is 

becoming increasingly obvious that the data from the sensors has more value if it can be 
digitized, recorded, exploited through enhancement and analysis, and transmitted to other 
locations. Currently this is being attempted primarily by cumbersome retrofit attachments that 
do not perform in an optimum manner such as clip on cameras and communications controllers 
external to the sensors. 

[0007] Examples of currently available surveillance devices are: Binoculars / Telescopes (e.g., 

0 Steiner 7x50G , 37 oz, 368 ft FOV at 1000 yards, 17 mm eye relief, Canon 15x45 IS Image 
Jg Stabilized Binoculars, 36 oz, 67 degree FOV, 15 mm eye relief, and other models); Image 
Cn Intensifier Devices (e.g., AN/PVS-10 Sniper Day/Night Sight (SNS) integrated day/night sight 
for the M24 Sniper Rifle which provides the sniper with the capability to acquire and engage 
~K targets under both day and night conditions. For nighttime operation, the system utilizes third 
generation image intensification technology); Passive, Third Generation Image Intensification 
yi (18mm Image Intensifier Tube); AN/PVS-14 Monocular Night Vision Device and Night Sight 
; if (The AN/PVS-4A is a night sight for an Individual served weapon which provides the soldier 
O with the capability to acquire and engage targets under night conditions. The system utilizes third 
generation image intensification technology); Passive, Third Generation Image Intensification 
25mm Image Intensifier Tube; AN/PVS-7D Night Sight (Provides leaders of combat infantry 
units with a light weight, Gen III night vision device for use in observation and command and 
control, and also can be mounted to small arms rail using a rail grabber); Passive third generation 
image intensification (18mm image intensifier tube, accepts all ancillary items of the AN/PVS- 
7D Night Vision Goggles); Uncooled FLIRs (e.g., AN/PAS- 13 TWS (Thermal Weapons Sight) 
manufactured by Raytheon, AN/PAS-20); Cooled FLIRs (e.g., HIT Second Generation FLIR 
manufactured by Raytheon); RF Imager; RF Probe; Laser Rangefinder (e.g., AN/PVS-6 
manufactured by Varo or LLDR-- Lightweight Laser Designator Rangefinder); NBC Detectors 
(e.g., Chemical Agent Detector by Graseby's Chemical Agent Monitor— a hand held instrument 
to monitor nerve and blister agents and can be reprogrammed to meet further threats); BRUKER 
IMS 2000 (uses the principle of ion mobility to differentiate between various agent vapors. 
Ambient air bearing water vapor in the form of natural humidity is drawn into the unit. And is 
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ionized by a low energy beta source. Different tracer gases enable detection of a range of gases 
as they pass through the membrane and react with ions); Radiation Detection (e.g., Radiac Set 
AN/VDR-2 for performing ground radiological surveys in vehicles or in the dismounted mode 
by individual soldiers as a hand-held instrument. The set can also provide a quantitative measure 
of radiation to decontaminate personnel, equipment, and supplies. Components of the Radiac Set 
include the Radiacmeter IM-243, Probe DT-616, and pouch with strap. Installation kits are 
available as common table allowances (CTA) items for installation of the Radiac Set in various 
military vehicles. The set includes an audible and/or visual alarm that is compatible with 
vehicular nuclear, biological and chemical protective systems in armored vehicles and also 
= interfaces with vehicular power systems and intercoms); Sensor Support Devices; Clip on Video 
-J3 Cameras; Still Video Cameras; Camcorders; GPS Receivers; Image Transmission Systems; 
y! PhotoTelesis MMR; Field Computers; Battery Packs; Tripods. 
[0CW8] The burdensome task of carrying and managing this amount of equipment in field 
^ operations is almost impossible. If field personnel are to be fully equipped and still remain 
s mobile and flexible, modularity, miniaturization and weight reductions are a necessity. 
m SUMMARY OF INVENTION 
[0809] The subject invention is directed to a compact, modular, comprehensive, multimedia, 
O surveillance system providing multiple data capture and management capability for field 
personnel. The system comprises a modular sensor array having a standard base module or 
platform upon which a myriad of systems may be mounted for providing a wide range of 
flexibility and functionality. 



[001tf|^jSv\ y On embodiment of the system comprises the base control module assembly, a daylight 
* v^ion assembly, a night vision assembly, a laser rangefinder, and a military GPS receiver. The 
daylight arid night vision configurations may be operated stand alone, or may be operated in 
conjunction with the PhotoTelesis MMR (Military Micro RIT (Remote Image Transceiver). 
Various Military Radios may be utilized for image and collateral data transmission. A remote 
base station cam be incorporated having a remote, possibly laptop, computer with appropriate 
Government protocol and/or commercial communications card and an image frame capture card, 
a printer, and a power inverter to operate the system on 24 VDC military power. The control 
module is adapted Vor transmitting all captured data to the base station via a wireless 
communications link or via plug-in cabling for archiving and managing the data, or may be 
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operated in conjunction with the PhotoTeleis MMR providing communications and processing 
functions. Additional modules include a high performance day vision system; an uncooled FLIR 
(Forward Looking Infrared sensor); cooled FLIR; RF probe; NBC detector; sensor computer 
modules; a laser rangefinder unit and the like. The system has full modularity and various 

components can be connected as desired, with virtually no limitation in the functionality. 

[001 1] \The system of the subject invention greatly improves the utility and functionality of field 

surveillance units. This is accomplished by providing a standard "platform" upon which different 
types of sensors and other components may be supported. This accomplishes many things, 
including the reduction in total electronics through shared utilization, use of one battery type, 
= providing a common user interface for all types of sensors, reduced field support costs and 
-J3 training requirements, and providing a single point of interface for data transfer of all data types 
fS to other systems using standardized formats and techniques, such as to vehicle systems, recording 
%1 systems, briefing systems, intelligence systems and the like. 
[001 2] In the preferred embodiment of the invention the base module contains the control panel, 

s the power source, electronics and a standardized mounting system. The various components are 
simply plugged into the standardized mounting system, whereby each component is properly 
W mounted and is connected to the power, electronics and control system. The changeover from 
p one component to another can be accomplished in a matter of a few seconds. By providing a 
standardized base module and compatible components a single power supply, single control 
electronics and single data management electronics, including imaging processing software may 
be used for all components, greatly reducing the weight and size of the overall surveillance 
system. 

[0013] It is, therefore, an object and feature of the subject invention to provide a fully modular, 

multiple function field surveillance system. 
[0014] It is another object and feature of the subject invention to provide a surveillance system 

having a single power supply, common control and management electronics. 
[0015] It is a further object and feature of the subject invention to provide a system capable of 

adjusting the gain of an image intensifier in conjunction with other functions for various modules 

of the system. 

[0016] It is also an object and feature of the subject invention to provide a system capable of 

permitting, in the alternative, viewing of "raw" or unprocessed video, or frame averaging, also 
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called frame integration, that allows accurate mathematical integration of multiple frames to 
provide enhanced images. 
[0017] Additional objects and features of the subject invention include: 

1) Shared use of image processing hardware and software for noise reduction for 
multiple component units (particularly useful for image intensifies and FLIRS). 

2) Shared use of image processing hardware and software for contrast enhancement 
for multiple component units. 

3) Shared use of motion detection and alarm hardware and software for multiple 
component units. 

tj3 4) Shared use of image stabilization hardware and software for multiple component 

2l units. 

U 5 

5) Shared use of contrast enhancement hardware and software for multiple 
12 component units. 

^ 6) Shared use of image cropping hardware and software for multiple component 

*\ units. 

2f 7) Shared use of image processing filtering functions for multiple component units. 

^ 8) Shared use of image compression hardware and software for multiple component 

units. 

9) Shared use of communications protocols, hardware and software for multiple 
component units. 

10) Shared use of digital storage hardware and software for multiple component 

units. 

11) Shared use of geolocation hardware and software, such as GPS, for multiple 
component units. 

1 2) Shared use of Geolocation hardware and software in conjunction with a magnetic 
compass, inclinometer and laser rangefinder in order to calculate geolocation of targets that are 
of interest. 

13) Shared use of power supply hardware and control software, and common battery 
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types for multiple component units. 

14) Shared use of video processing hardware and associated software, such as the 
video decoder and video encoder circuits, video time base, and the like. 

15) Shared use of Gain Control Elements for multiple optical imaging modules. 

1 6) Shared use of video zoom hardware and software for multiple component units. 

17) Shared use of Electronic Viewing Device for multiple component units. 

1 8) Shared use of user interface controls for multiple component units. 

19) Shared use of a handgrip for portable use of multiple component units. 

20) Shared use of electronic interface for sensor data to other systems for multiple 
component units. 

21) Shared use of mounting equipment for multiple component units, such as a tripod 
mount or leg kit. 

22) A common mechanical and electrical method of attaching various sensors to a 
control module and for providing support and electrical interface. 

23) A common user interface with similar commands for similar functions between 
multiple component units. 

24) Use of an attachable daylight camera module on a common control module. 

25) Use of an attachable image intensifier module on a common control module. 

26) Use of an attachable uncooled FLIR module on a common control module. 

27) Use of an attachable cooled FLIR module on a common control module. 

28) Use of an attachable RF probe on a common control module. 

29) Use of an attachable RF imaging sensor on a common control module. 

30) Use of an attachable laser rangefinder on a common control module. 

31) Use of an attachable chemical detection and analysis module on a common 
control module. 

32) Use of an attachable radiation detection and analysis module on a common 
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control module. 

33) Use of a thermonic cooler to cool a focal plane array FLIR - approximate 
temperature 77°K. 

34) Storage of sensor setting parameters in non-volatile memory in the sensor module 
(gain, integration, contrast, and the like). 

35) Dynamic menus in control module changes with sensor that is attached. 

36) Downloading of control module code or commands, or part of the code or 
commands from the Sensor to the control module. This provides a "universal" control module 
that can support sensors developed after the control module, or upgrades to sensors developed 
after the control module. 

37) Use of an "http" browser in the control module. 

38) Use or "mini-servers" to serve the user interface and application for the sensor. 

39) Use of IP as an interface between the control module and the sensor. 

40) Use of IP as an interface between the control module and other devices. 

41) Use of a "mini-server" in the control module to serve data to workstations or 
other devices from the sensor to the other devices. 

Other objects and features of the invention will be readily apparent from the accompanying 
drawings and description of the preferred embodiment. 



BRIEF DESCRIPTION OF THE DRAWINGS 



[0018] 



Fig. 1 is an exploded view of the system including a base module, a plurality of sensor 
components, connectivity modules and communications modules. 



[0019] 



Fig. 2 is similar to Fig. 1 with a military sensor computer base module. 



[0020] 



Fig. 3 is a perspective view of the computer base module of Fig. 2, showing the mounting 
rail for the sensor components in greater detail. 
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[0021] Fig. 4 is a view similar to Fig. 3, showing the day channel module and the night channel 

module. 

[0022] Fig. 5 is a flow chart of the system electronics in the base module. 

[0023] Fig. 6 shows the elements of an image intensifier module. 

[0024] Fig. 7 shows sequencing adjustment for the image intensifier module. 

[0025] Fig. 8 illustrates programmable elements of the system adapted for adjustment in any 

desired manner for each step in gain setting. 

[00J6] Fig. 9 is a depiction of the operation of the hardware and state machine for the frame 
m averager. 

[0(S7] Fig. 10 is a block diagram of the base module assembly. 

[0028] Fig. 10a is an exploded view of the base module assembly. 

[0i|9] Fig. 10b is a partial view of the hidden portion of the exploded view of Fig. 10. 

[0B30] Fig. 10c is a perspective view of the assembled unit. 

[0031] Figs. lOd-lOi are circuit diagrams for the base control module. 

[0032] Fig. 1 1 is a block diagram of the night vision channel module. 

[0033] Fig. 1 la is an exploded view of the night vision channel module assembly. 

[0034] Figs. 1 lb-1 Id are circuit diagrams for the night vision controller. 

[0035] Fig. 12 is a block diagram of the day vision channel module. 

[0036] Fig. 12a is an exploded view of the day vision module assembly. 

[0037] Figs. 12b-12e are circuit diagrams for the day vision controller. 
DESCRIPTION OF THE PREFERRED EMBODIMENT 
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[003^ \ An exploded view of the modular system of the subject invention is shown in Fig. 1 . The 

iD^se module 10 includes the electronics (inside); controls 12 and power supply 14 for all of the 
confponents of the system. In this embodiment, the module includes a standard connector (not 
visible^for cabling the module a management unit such as, by way of example, the PhotoTelesis 
MMR 1 5v The MMR unit includes standard connectors 1 6, 1 8, and 20 for a communications link 
22, an inp^it device such as the keyboard 24 and a breakout box 26, respectively. The 
communicat\ons device in the preferred embodiment is a PSC-5 with a Sincgars radio. It should 
be understood that other communications links such as cellular telephone, secure telephone, 
satellite transmission, an Internet gateway or other could be substituted without departing from 
^ the scope and spirit of the invention. The input device is shown as a ruggedized keyboard. Other 

i L-i \ 

\0 input devices cantoe readily substituted. The breakout device 26 is adapted for further increasing 

Sj the flexibility of the system by permitting the attachment of additional modules such as, by way 

JJJ of example the PLfflR unit 28 and MELIOS unit 30. 

[0©9] The base module 10 includes a mounting rail system 32, as better seen in Figs. 2 and 6. 

p The mounting rail system defines a channel or slide for receiving the compatible connector rail 

Sj 1 34 provided on each ofxhe various sensor units, as here shown including the high performance 

y3 day module 36, the laser hange finder 38, the high performance night module 40, the uncooled 

2 FLIR module 42, the FLIRVnodule 44, the RF probe module 46, and the NBC detector 48. 



[0040] The base module and each of the component modules also include a mechanical locking 

system for locking the installed module on the base. In the illustrated embodiment, the base 
includes a threaded hole 50 and each of the components include a mated locking screw 52 for 
securing mounted component to the base once the rail 34 is received in the slide channel 32. 



[004] 




A common connector plug assembly 54 is provided on each of the component modules 
■hid i&received in a mated receptacle 56 on the base 10 as the component is received in the slide 
and locked in mounted position. This connects the module with the power supply, controls and 
system electronics. The receptacle 56 may also be used for connecting various connector cables 
to standard video or other devices, such as the monochrome RS- 1 70 cable 58, the switcher cable 
60 and the color RS-170 cable 74, each of which is provided with the compatible plug 54. 



[0042] 



The system shown in Fig. 2 is similar to that shown in Fig. 1, the base module 10 and the 



MMR module 
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5 and keyboard 22 having been replaced by a handheld military sensor computer 
62 having a Hinged keyboard input device 64 and a display panel or screen 66. The 
communication s link 22 is attached directly to the computer by cabling, as previously described. 

nponents are mounted on the slide 32, as before, with the locking system and 
assembly provided on the computer base in the same manner as the base unit of 



The various C0j 
connector plug 
Fig. 1. 



[0043] The computer base is shown in greater detail in Figs. 3 and 4. With specific reference to 

Fig. 3, the component mounting slide channel 32 is mounted on the top side of the computer base 
unit 62. The connector receptacle 50 in this embodiment is located in the rearward end of the 
slide channel for receiving the compatible plug 54 in the various modules, see Fig. 4. The 
jff locking assembly is also located in this position. The eyepiece or eyecup 70 and the viewfinder 
EH 72 extend conveniently rearward of the unit. As best seen in Fig. 4, a component rail system 34 
is axially positioned for sliding in the channel 32 until the component plug 54 mates with the base 
^ receptacle 52, after which the locking system is tightened for locking the component in functional 
D assembly with the base. The same system is utilized in the configuration of Fig. 1. 

Fig. 5 is an overall system control diagram for the modular system of the subject 
invention. Each sensor module is adapted to be mechanically secured to the base via the 
previously described rail and slide system 32, 34, as indicated. When this is completed, the 
receptacle and plug system 52, 54 completes the electrical interface connection, permitting output 
signals to be transmitted from the module to the base via line 80, control signals to be transmitted 
from the base to the module via line 82 and power to be supplied via power line 84. The control 
processor is a low power embedded processor 86. The preferred embodiment of the invention 
includes an image stabilization sensor 88, a magnetic compass 90 an inclinometer 92 and a GPS 
receiver 94 all housed within the base unit and connected to the control processor 86 for assisting 
in the collection of useful data by stabilizing, aiming, positioning and managing the collected 
data. A common power supply 96 is provided and may use external power input via cabling 97 
or an integral battery source 98. The various inputs to the processor 86 include the sensor input 
as well as the various managing inputs from devices 88, 90, 92 and 94. 
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[0045] The sensor input 80 is first introduced into a video switching and format conversion 

circuit 100 for both encoding and decoding the raw data. This circuit is in communication with 
a real time image processing circuit 102 which is controlled and managed by the central 
processor 86 for providing output via the circuit 100 to the viewfinder network 104, external 
video output signals as indicated at 1 06, and communication links with various video devices as 
indicated on lines 108 and 1 10 at interface 112. The control processor 86 also controls the user 
display controller 114 and is in communication with the user input device 116 (such as the 
keyboard 24 shown in Fig. 1). Network interfacing circuit 118 provides communication over 
networks via the interface 112. Likewise, an input/output control module 120 provides external 
_ control via the interface 112 and communication links are provided through the interface 1 12 via 
y3 the communications processor, all controlled by the central processor 86. 
ff% 

[0046] The communications processor and software of the control system is adapted to include 

^ the following functions: 

I* 1) Shared use of Image Processing hardware and software for noise reduction for 

Iji multiple component units (particularly useful for Image Intensifiers and FLIRS). 

lJJ 2) Shared use of Image Processing hardware and software for contrast enhancement 

r: for multiple component units. 

3) Shared use of Motion Detection and Alarm hardware and software for multiple 
component units. 

4) Shared use of Image Stabilization hardware and software for multiple component 

units. 

5) Shared use of Contrast Enhancement hardware and software for multiple 
component units. -y 

6) Shared use of Image Cropping hardware and software for multiple component 

units. 

7) Shared use of Image Processing Filtering Functions for multiple component units. 

8) Shared use of Image Compression hardware and software for multiple component 

units. 
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9) Shared use of Communications Protocols, hardware and software for multiple 
component units. 

10) . Shared use of Digital Storage hardware and software for multiple component 

units. 

11) Shared use of Geolocation hardware and software, such as GPS, for multiple 
component units. 

1 2) Shared use of Geolocation hardware and software in conjunction with a magnetic 
compass, inclinometer and laser range finder in order to calculate geolocation of targets that are 
of interest. 

1 3) Shared use of Power Supply hardware and control software, and common battery 
types for multiple component units. 

14) Shared use of Video Processing hardware and associated software, such as the 
video decoder and video encoder circuits, video time base, and the like. 

15) Shared use of Gain Control Elements for multiple optical imaging modules. 

1 6) Shared use of Video Zoom hardware and software for multiple component units. 

17) Shared use of Electronic Viewing Device for multiple component units. 

18) Shared use of User Interface Controls for multiple component units. 

19) Shared use of a Handgrip for Portable Use of multiple component units. 

20) Shared use of Electronic Interface for sensor data to other systems for multiple 
component units. 

21) Shared use of Mounting Equipment for multiple component units, such as a tripod 
mount or leg kit. 

22) Common Mechanical and Electrical Method of attaching various sensors to a 
control module and for providing support and electrical interface. 

23) A Common User Interface with similar commands for similar functions between 
multiple component units. 

24) Use of an attachable Daylight Camera Module on a common Control Module. 
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25) Use of an attachable Image Intensifier Module on a common Control Module. 

26) Use of an attachable Uncooled FLIR Module on a common Control Module. 

27) Use of an attachable Cooled FLIR Module on a common Control Module. 

28) Use of an attachable RF Probe on a common Control Module. 

29) Use of an attachable RF Imaging Sensor on a common Control Module. 

30) Use of an attachable Laser Rangefmder on a common Control Module. 

31) Use of an attachable Chemical Detection and Analysis Module on a common 
Control Module. 

32) Use of an attachable Radiation Detection and Analysis Module on a common 
Control Module. 

33) Use of a Thermonic cooler to cool a focal plane array FLIR - approximate 
temperature 77°K 

Thermonic cooler from Eneco, Inc. or equivalent 

Focal Plane Array from DRS Technologies or equivalent 

Packaging that will encompass the Thermonic Cooler and Focal Plane 
Array such that the cryogenic temperatures can be maintained with minimal thermal loss and 
energy consumption. 

34) Storage of sensor setting parameters in non-volatile memory in the sensor module 
(gain, integration, contrast, and the like) 

35) Dynamic Menus in Control Module changes with sensor that is attached. 

36) Downloading of Control Module code or commands, or part of the code or 
commands from the Sensor to the Control Module. This provides a "universal" control module 
that can support sensors developed after the Control Module, or upgrades to sensors developed 
after the control module. 

37) Use of an http browser in the Control Module. 

38) Use or "mini-servers" to serve the user interface and application for the sensor. 
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39) Use of IP as an interface between the control module and the sensor. 

40) Use of IP as an interface between the control module and other devices. 

41) Use of a "mini-server" in the Control Module to serve data to workstations or 
other devices from the sensor to the other devices. 

[0047] Another important aspect of the invention is the method of combining one or more basic 

user controls to perform optimized adjustments of multiple gain elements in the various 
components, particularly the night vision system. This can be applied to the Image Intensifier 
module, the uncooled FLIR module, or to the cooled FLIR module in a similar fashion. There 
are a multitude of programmable gain elements in the complex system that can adjust gain. In 

O 

(i g many cases, increasing the gain of an element will increase the noise from that element. A 

$i? notable exception might be an iris. However increasing the gain of an image intensifier tube to 

y = 

Uj the maximum will likely cause the noise of the tube to increase. Unfortunately, if an image 
gi intensifier tube is run such that its gain is low, it is photon starved, the output would be low. 
J" Thus, to get an image through the system the gain of the camera may be increased substantially 
3 and this will generate noise. The key is to balance all of the mechanical, electro-optical and 
fy electronic elements such that each element of the system is running at an optimum gain for the 

^- -incoming light level, and that each element is feeding the next element at an optimum level. 
[0©48]5^ y "rig. 6 shows the a graphic of the relationship of the elements of an image intensifier 
odule fqr the base unit. The left vertical axis, as drawn, is light intensity. The horizontal axis 
is gain. Starting on the left, we show a scene to be imaged. The lens contains a motorized iris 
that can be utilized to mechanically control the amount of light that is projected on the input side 
of the image intensifier tube. The iris ideally is at stepped down to various partial open positions 
under under bright conditions and opened up under low light conditions, thus providing a more 
uniform illumination level to the intensifier tube under varying light levels. 
[0049] The image Wensifier is a gated type of tube with and external gain input control. This 

control in prior art systems is a simple variable resistor that is adjusted by the user while viewing 
the output of the tube\ In the system of the preferred embodiment it is controlled by a circuit 
element that is interfaced to the control processor 86 of the base unit (see Fig. 5). Therefore, the 
gain of the tube can be admsted under computer control. The relay lens then images the light 
coming from the output sine of the image intensifier to a solid state camera such as, by way of 
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example, a CMOS camera on CCD camera. The camera has a gross light level adjustment that 
is called the shutter. This is an electronic mechanism that gates the active area of the CMOS or 
CCD sensor for a specific afrnount of time, thus letting photons discharge wells or electrically 
controlled gates in the solia state array. The longer the time that is metered to the imager chip 
for exposure, such as 1/30/of a second, the more sensitive the imager will be to the light. The 
shorter time that the light Is metered to the chip, such as 1/2000 or 1/10000 of a second, the less 
sensitive that the camerawill be to the light, thus controlling the camera gain. 

[0050] The camera also has highly sensitive analog preamplifiers that take the measured photon 

signal and amplify the electronic signals resulting from the photon interaction with the 
y semiconductor. The gain of this amplifier can be controlled in the analog domain under digital 
CO control from the processor. 

[0||^1] Various filters can be programmed in and out based upon various modes of operation. 

" For example, if the tube is being operated at a higher gain, it may produce noise that can be 
l_ filtered in the analog domain. This function can just as easily be implemented in the digital 
{S domain by, for example, a high speed DSP, but the filter element would then be located after that 
AID converter. Finally, look-up tables can be utilized to process incoming video signals to 
D enhance contrast, brightness or provide other non-linear adjustments such as gamma. 

[0052] All of these adjustments can be provided in sequence, such as is illustrated by Fig. 7. In 

this case the elements would be operated at a nominal low gain position when a high level of 
light is provided. As the light level decreases the gain of each stage is brought up to maintain 
image brightness and contrast at the output of the sensor system. As each stage has the gain 
increased, a corresponding noise increase will likely be seen. Stage-by-stage, each is turned up 
until maximum gain is achieved. Individual gains may be adjusted in a linear or non-linear 
fashion. Linear adjustment is shown for simplification. The individual elements can be adjusted 
in either linear or non-linear manners, and can be calculated by mathematical functions in 
software or by look-up-tables. 

[0053] "^^^^^8* ^ iH ustrates a more flexible method wherein all programmable elements of the system 
{y be adjusted in any desired manner for each step in gain setting. In this method, one or more 
of the programmable elements can be adjusted for each and every step increase in gain. The 




[0§54] 



[0056] 



individual elements 
calculated by mathem 
to Fig. 8, the scene is 



there to a filter system 
by the look-up tables 
the frame integrator i 
line out 152, and varib 
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can be adjusted in an either linear or non-linear manner, and can be 
tical functions in software or by look-up-tables. With specific reference 
sho wn as the individual designated by the reference numeral 126. This is 
picked up by the image intensifier tube 128 and directed through the lens assembly 130. The 
intensifier tube include s an irised lens 133 controlled by the gain module 132. The lens assembly 
130 is focused on the : olid state camera chip 134. A timebase module 136 controls the shutter 
speed. The image ou put from the camera chip is introduced into an analog camera 138 from 



140. It is then converted to a digital signal at convertor 142 and modified 
44, afterwhich it is introduced into the fame integrator 146. The output of 
distributed to an analog output line through the convertor 148, a digital 
us other components such as the viewfinder 154. 



I 

One of the primary functions of the Common Control Module is the video frame 
averaging to remove video noise generated by the camera and video amplifiers used in the image 
intensifier, uncooled FLIR and cooled FLIR units. In the Intensifier, noise is also generated by 
the image intensifer tube, which is removed in addition. 

A detailed diagram of the video frame averager is shown in Fig. 9. A primary design goal 
of the video frame average is low power and small size. To accomplish this the frame average 
design is based around a single FPGA and a dynamic memory operated in conjunction with a low 
power video A/D and D/A. The averaging function can also be implemented with a high speed 
digital signal processor (DSP), however the power consumption, size, weight and cost would be 
greater and would impact the user. 

The frame averager implementation in the preferred embodiment has two modes: 1) 
video bypass that allows viewing of "raw" or unprocessed video, and 2) "frame averaging", also 
called "frame integration", that allows accurate mathematical integration of two, four, eight, or 
sixteen video frames. More frames could be integrated with the addition of memory, a larger 
counter, and a larger accumulator/barrel shifter. The frame averager of the preferred embodiment 
has enough memory to store all pixels from previous frames in the amount of the number of 
frames to be integrated. For example, if integration is desired for 16 frames of 720 pixels by 440 
pixels, sufficient memory is provided to store the raw data for 16 frames of 3 16,800 pixels. This 
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is utilized to calculate the average on a basis that is updated on a real-time frame rate, such as 30 
frames per second. 

[0057] The basic video averaging algorithm used consists of the following primary steps: 

INITIALIZE 

Zeroize all Memory Locations 
Clear Frame Address Counter 
Clear Pixel Address Counter 

G 

05 Clear Accumulator 

H: Next Pixel 

=P Select Zero 

m Story in Memory 

;jr Increment Pixel Counter, Check for Last Pixel Location 
If No, Go to Next Pixel 

Increment Frame Counter, Check for Last Frame 

If No, go to Next Pixel 

Reset Frame Counter 

Go to Next Pixel 

AVERAGE 

Barrel Shift Accumulator 
Output Average Pixel Value 
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Read Oldest Pixel Value at Pointer 
Subtract from Accumulator 
Input Next Pixel Value 
Add to Accumulator 

Increment Pixel Counter, Check for Last Pixel 
If No, go to Average 

Increment Frame Counter, Check for Last Frame 
If No, go to Average 
Reset Frame Counter 
Go to Average 

Fig. 9 is a depiction of the operation of the hardware and state machine implemented in 
the preferred embodiment in the base control module. Video from the sensor element is presented 
on input 501 to the video decoder chip 502. The decoder chip separates out the timing signals 
such as horizontal sync, vertical sync, blanking, and a pixel clock utilizing sync stripping and 
phase lock loop circuits in a well known manner. It also contains an analog-to-digital converter 
504 to convert the analog video input to a digital signal for processing. It is also evident that a 
digital signal can be directly input to the system by bypassing the video decoder chip 502. 

The frame averager incorporates bypass mode whereby the incoming video can be routed 
the output without averaging. This is accomplished by setting a bypass command into control 
iter 535 utilizing data bus 536 and write strobe 535. This sets the signal on wire 573 to allow 
essed video data 506 to be selected by multiplexer 507 input B to be presented to the D/A 
on the video encoder chip 509 to produce the unprocessed video at video output 550. 

The frame averager is set into the video averaging mode by setting the control register 
534 to average by presenting a command on the data bus 536 with a strobe on 535, and by setting 



# 
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the number of frames to be averaged by setting the register 538 by presenting a value on bus 536 
with a strobe on 539. This value is utilized to determine the divisor being done in the barrel 
shifter 517. This also determines the number of memory positions utilized in memory 522 by 
setting the range of the fame counter 532. 

[006lJ|\ The frame average is first initialized by zeroing memory 522. The state machine 524 

first selects input A on multiplexer 520, which is a zero valueThis presents data of zero value to 
tne din on memory 522. The frame counter 532 and pixel vounter 560 are reset by a pulse on 
53(a The memory then is written with a strobe on "Write" via wire 53 1, then them pixel address 
is incremented to the next location with a pulse on 561. The process of writing is repeated until 
^ the appropriate number of memory locations for one full frame have been zeroed. Then the 
ID frame counter 532 is incremented by a pulse on 529, and the above process of writing all pixels 
m in a framAis repeated. This continues until all frames as is specified by register 538 have been 
it zeroed. This leaves all necessary memory locations set at zero. The accumulator 5 1 3 is reset by 
+= signal 530 frqm the state machine. The state machine 524 can then exit the initialization process. 

[TO62] Note that the above initialization process can be skipped and the memory with an 

appropriate number of cycles will self initialize. This, however, can result in a "glitch" in the 
H video each time that the video averager is activated on, which may be objectionable. Use of the 
initialization process will gracefully start up the integrator. 

[0063] Another method to initialize the integrator without generating a video "glitch" is by 

holding off the switching of the mltiplexer 507 from B, live video, to A, averaged video, until 
after the integrator is fully initialized by processing the full number of pixels for a total number 
of fields specified by the Average register 538 before switching the mltiplexer 507 from the B 
input to the A input. This masks the average stabilizing by presenting live video until the 
averager has process one completed sequence of frames and thus is holding one complete set of 
historical data in memory. 

[0064] After initialization, the video averager can be fully activated. The multilpexer 507 is set 

for the A input which allows averaged video to pass to the video output 550 via the video encoder 
549. Multiplexer 520 is set for the B input which allows video pixel data to be written to the 
memory 522. 
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[0065] Multiplexer 51 1 is set to the A input by state machine 524. New pixel data on bus 506 

is selected to go to the adder 513, which also may be called the accumulator. The barrel shifter 
5 1 7 is set to divide by the integer amount specified in register 538 by doing a binary shift by the 
appropriate number of bit for division: 1 bit for averaging 2 frames, 2 bits for 4 frames, 3 bits 
for 8 frames, 4 bits for 16 frames, and so on in a binary power progression. Multiplexer 520 is 
set to input B during averaging operation to allow the actual video pixel value to be stored in 
memory by the control signal 528. 

[0066] The above having been set, the averaging process then follows the repetitive processes. 

Memory Chip 515 contains the averaged field at any one time. There is a value for each pixel 
= of the field stored in the memory, and in the preferred embodiment it is of higher precision than 
CO the incoming video as it has not been divided until it is processed by the barrel shifter 517. 

■f©{§£7] I \ A historical copy of all of the pixel date from the last N frames that have been averaged 
yrSy vs stored in memory 522. The sum for each pixel in memory 5 1 5 is updated by selecting the raw 
pixel date for the oldest frame in the memory, subtracting it from the previous sum in acumulator 
m memory $J5 then adding in new pixel data form bus 510. This is accomplished by setting the 
^ multiplexer^ 1 1 to input B by using signal 525, setting the adder 5 13 to subtract by using signal 
O 526, then capturing the result by clocking memory 515 with a clock on 527. The multiplexer 5 1 1 
is then set to input A allowing the newest incoming pixel data 5 1 0 to be summed into the average 
by setting the adder 513 to add utilizing wire 514, then clocking the memory 515 to capture the 
new sum. This sum is then barrel shifted using barrel shifter 5 1 7 to do the divide, and presented 
to the video encoder Via bus 518 and multiplexer 507. The averaged video data is then available 
on video output 550. 'This process is repeated for each pixel in the frame, with the pixel counter 
560 being incremented bwhe state machine 524 for each pixel clock. After the average has been 
processed for one frame, tnk state machine increments the frame counter 532 via signal 529, or 
resets it via signal 530 if the Dotal number of frames to be integrated has been met. 

[0068] It can be seen that the accumulator memory 515 can be implemented within in memory 

522 to conserve hardware, but the accuracy of the accumulator is grater than the historical data 
so the memory either has to be made wider in word width, or two memory cycles are required. 
In addition moving the accumulator int(Mie memory 522 places an additional bandwidth burden 
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on the memory, this causing it to have to be an extremely expensive fast part and causing it to 
consume more power. In the preferred embodiment it was found that maintaining separate 
memory for the accumulator and the historical memory is preferable. 

[0069] Figs. 10 and lOa-lOi are detail drawings of the base module shown in Fig. 1. Fig. 10 is 

a block diagram showing the major components and their interconnect! vity. Specifically, the 
base module includes a base unit coupled to a battery support assembly, an electronic view 
finder, a PWA EMI filter, and input or keypad assembly for controls and a PWA sensor interface. 
In addition a power supply assembly and a video frame averager is provided. The specific 
mechanical components of the assembly are shown in the exploded view of Figs. 10a and 10b 
Jq and assembly of Fig. 10c. The assembly is self explanatory from the drawing. For references 
purposes, the components are numbered as follows: 

%P 201 top assembly of the control module; 

7~ 202 hand strap 

%l 203 handgrip 

□ 204 O ring 

205 battery support 

206 support 

207 not used 

208 eyecup 

209 not used 

210 retainer for keypad module 

211 top cover 



housing weldment 



213 P WA sensor interface 

214 PWA EMI filter 

215 not used 

216 pan screw 

217 silicone sealant 

2 1 8 screw 

219 washer 

220 batter cap assembly 

22 1 not used 

222 not used 

223 elastic lock nut 

224 not used 

225 dowel pin 

226 support leg 

227 battery interface module cap 

228 battery sensor cap 

229 gasket 

230 ribbon 

23 1 cable 

232 retainer cap 
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233 flat head screw 

234 serial number plate 

235 keypad 

236 locking ring 

237 video frame processor and control board 

238 power supply control board 



y 239 adhesive 



m 240 tape 

Ijj 241 viewfinder cable assembly 

[0J70] The control circuitry is shown in Figs. lOd-lOi. All of the pin numbers are those of the 

Ln manufacturer. 

pj)71] Figs. 1 1 and 1 la- 1 Id are detail drawings of the night vision module shown in Fig. 1 . Fig. 

^ 1 1 is a block diagram showing the major components and their interconnectivity. Specifically, 
the night vision module includes a base unit having a controller with an MS connector, an 12 tube 
and a commercial camera and lens assembly. The MS connector connects the module to the base 
when locked in the receiving slide and rail system previously described. The specific mechanical 
components of the assembly are shown in the exploded view of Fig. 11a. The assembly is self 
explanatory from the drawing. For references purposes, the components are numbered as 
follows: 

301 top assembly 

302 hand grip 

303 O ring 



304 



slide 
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305 rear end cap 

306 front end cap 

307 not used 

308 camera support 

309 weldment 

310 j ackscrew retainer 

311 not used 

312 ring 

313 washer 

314 flat head screw 

315 connector cap 

316 sealant 

317 tape 

318 adhesive 

319 not used 

320 not used 

32 1 compression spring 

322 12 tube assembly with relay lens, camera and interface board 

323 locking screw 

324 locking screw 



325 washer 

326 not used 

327 Allen screw 

328 splitlock washer 

329 screw 

330 pan head screw 

331 not used 

332 pan head screw 

333 standoff 

334 standoff 

335 standoff 

336 PWA controller 

337 PWA contact 

338 PWA peak detector 

339 Schrader valve 

340 Ring stopper 

341 CCD camera assembly 

342 objective lens 

343 relay lens 

344 set screw 
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345 O ring 



346 set screw 

347 relay lens support 

348 serial no. plate 

349 not used 

350 control cable assembly. 

4f The night vision circuitry is shown in Figs. 1 lb-lOd. All of the pin numbers are those of the 
03 manufacturer. 

01 

[QJ|72] Figs. 12 and 12a-12e are detail drawings of the day vision module shown in Fig. 1. Fig. 

"2f 12 is a block diagram showing the major components and their interconnectivity. Specifically, 

the day vision module includes a base unit having a controller with an MS connector, and a 
yfl commercial camera and lens assembly. The MS connector connects the module to the base when 
"'i locked in the receiving slide and rail system previously described. The specific mechanical 
O components of the assembly are shown in the exploded view of Fig. 12a. The assembly is self 

explanatory from the drawing. For references purposes, the components are numbered as 

follows: 

401 top assembly 

402 hand grip 

403 slide 

404 O ring 

405 lens 

406 lens cap 

407 end cap 



408 not used 

409 retainer 

410 support 

411 weldment 

412 jackscrew 

413 not used 

414 adhesive 

415 washer 

416 ring 

417 flat head screw 

418 connector cap 

419 not used 

420 compression spring 

421 lens 

422 self-locking screw 

423 camera 

424 PWA controller 

425 Schrader valve 

426 retainer 

427 pan head screw 
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428 standoff 

429 screw 

430 washer 

43 1 Allen screw 

432 set screw 

433 pan screw 

434 serial number plate 

435 O ring 

436 interface cable assembly 

437 video control cable assembly 

438 power cable assembly 

439 communications cable assembly 

440 sealant 

44 1 splitlock washer 

The day vision circuitry is shown in Figs. 12b-12e. All of the pin numbers are those of the 
manufacturer. 

While certain features and embodiments of the invention have been described in detail 
herein, it will be readily apparent that the invention includes all modifications and enhancements 
within the scope and spirit of the following claims. 



